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INTRODUCTION 


The  following  narrative  provides  a  response  from  the  State  of  Missouri  to  requirements  established  by  the  U.S. 
Department  of  Labor  -  Employment  and  Training  (DOLETA)  to  provide  an  Annual  Report  on  the  activities 
funded  and  implemented  by  the  Workforce  Investment  Act  (WIA),  Public  Law  105-220.  As  such,  this  Annual 
Report  is  prepared  with  the  content  and  format  of  the  minimum  required  elements  for  staff  within  the 
Performance  Unit  of  DOLETA.  While  this  information  is  certainly  available  and  may  be  used  for  public 
consumption,  it  is  not  intended  to  be  a  public  accounting  to  the  stockholders  of  the  State  workforce 
investment  system.  It  is  for  reference  by  the  Performance  Unit  staff  of  DOLETA. 

Missouri's  Workforce  Performance  Reporting  Process 

The  WIA  is  administered  by  the  Missouri  Division  of  Workforce  Development  (DWD),  a  division  of  the 
Department  of  Economic  Development.  The  workforce  investment  system  recognized  by  the  public  is 
composed  of  the  Missouri  Career  Centers,  distributed  statewide,  and  part  of  the  American  Job  Center 
network.  Customers  may  also  access  the  system  on-line  at:  http://iobs.mo.qov.  The  following  graphic  (Graph 
1,  page  4)  shows  Missouri's  Career  Center  performance  reporting  process. 

Career  Center  customers  access  the  system  either  on-line,  where  they  can  perform  a  partial  registration  and 
obtain  information  about  Missouri  Career  Center  services  and  programs,  or  in  person  where  they  can  receive 
staff  assistance  to  register  for  program  participation  or  other  services.  For  both  types  of  access  the  process  is 
supported  by  Missouri's  Toolbox  Enterprise  case  management  system.  The  word  "enterprise"  embodies  the 
function  and  the  concept  that  this  software  does  it  all;  enables  customer  registration,  tracks  information  for 
case  management,  and  "populates"  a  computer  file  that  is  a  database  permitting  the  creation  of  performance 
reports. 

The  database  that  Toolbox  generates  is  exactly  the  same  as  the  WIA  Standardized  Record  Data  (WIASRD)  base 
that  is  required  by  the  USDOL  to  be  used  for  performance  reporting.  The  WIASRD  contains  individual 
participant  data  on  their  characteristics,  the  activities  completed,  and  their  program  outcomes.  Graph  1 
indicates  that  Toolbox  and  the  WIASRD  are  used  to  create  preliminary  reports. 

Missouri  uses  customized  software  provided  from  FutureWork  Systems,  a  web-based  application  service 
which  helps  States  and  Local  regions  manage,  track,  report  and  analyze  WIA  performance  data.  The  software 
which  is  called  MoPerforms  permits  the  analysis  of  each  element  in  the  WIASRD  by  variables  which  are  not 
required  by  DOLETA  for  performance  reporting.  This  generates  the  preliminary  performance  reports,  but  also 
allows  special  queries  about  the  data  to  be  made  beyond  the  basic  minimum  reporting  requirements.  This 
information  is  then  used  for  a  wide  range  of  program  adjustments  and  performance  management  decisions. 
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Graph  1:  Missouri's  Worl<force  Performance  Reporting  Process 
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Reports  at  the  preliminary  stage  are  widely  distributed  each  quarter  among  management  staff  and  then  the 
process  becomes  one  of  report  verification  and  evaluation  which  incorporates  the  JobStat  team.  JobStat  is  a 
team  approach  used  in  Missouri  to  ensure  there  is  a  WIA  performance  reporting  subject  matter  expert  (SME) 
in  each  of  Missouri's  fourteen  workforce  investment  regions.  JobStat  SMEs  are  the  front  line  of  defense  for 
reviewing  the  preliminary  quarterly  reports,  reporting  results  and  concerns  to  regional  managers,  and  for 
participating  in  quarterly  JobStat  meetings  that  are  used  to  monitor  issues  and  discuss  performance 
management  strategies.  Reports  are  submitted  to  DOLETA  on  a  quarterly  basis,  only  after  extensive  staff 
verification  and  JobStat  vetting. 

In  September  of  2014,  the  Annual  Report  in  data  format  was  submitted  to  DOLETA.  The  sanctioning  process 
was  also  initiated  at  this  time  for  any  regional  performance  for  which  the  measurable  goals  were  not  achieved 
In  compliance  with  DOLETA  policy  and  regulations,  any  region  achieving  less  than  80%  of  their  negotiated  goal 
on  any  common  measure  becomes  subject  to  a  sanction  action.  If  the  measure  is  not  achieved  for  a  given 
year,  each  region  is  required  to  develop  a  Technical  Assistance  Plan.  For  any  measure  missed  for  the  second 
consecutive  year,  the  region  is  required  to  submit  a  Performance  Improvement  Plan. 

Finally,  the  Data  Element  Validation  (DEV)  process  is  implemented,  as  required  and  according  to  instructions 
provided  by  DOLETA  typically  completely  at  the  same  time  as  any  necessary  sanctioning.  The  DEV  process 


Missouri  WIA  Annual  Report  for  DOLETA  -  Program  Year  2013 


4 


involves  reviewing  samples  of  participant  records  against  source  documentation  to  ensure  compliance  with 
federal  definitions.  Missouri  also  benefits  from  the  extensive  "enterprise"  nature  of  Toolbox  in  that  many 
data  validation  tasks  use  a  "desk  monitoring"  level  review  which  are  partially  completed  by  on-site  validation 
auditing  of  client  files. 

The  following  narrative  of  this  report  "unpacks"  each  of  the  performance  reporting  steps  shown  in  Graph  1 
and  explains  Missouri's  compliance  with  the  required  elements  for:  Assurance  of  uniform  performance  report 
preparation;  Reporting  the  core  and  customer  satisfaction  measures  attained  for  the  Program  Year; 
Reflecting  on  the  costs  analysis  of  program  activities  relative  to  their  effect  on  performance;  Status  of  state 
evaluation  activities;  and  Implementation  and  effective  use  of  program  administrative  waivers. 


Assurance  of  Uniform  Reporting 

Missouri's  Toolbox 

As  previously  explained,  the  Toolbox  software  system  was  designed  by  Information  Technology  professionals 
that  were  thoroughly  familiar  with  WIA.  As  such,  it  effectively  mirrors  the  WIASRD  data.  This  is  a  sound 
assurance  that  all  required  data  elements  are  reported  uniformly  so  that  a  state-by-state  comparison  can  be 
made  among  various  states. 

Changes  to  the  WIASRD  are  maintained  at  all  times  also  as  changes  to  Toolbox. 
JobStat 

As  stated  earlier,  Missouri  uses  a  JobStat  workgroup  comprised  of  SMEs  from  each  of  the  fourteen  workforce 
regions.  This  group  is  provided  staff  support  by  DWDs  JobStat  unit  that  is  continuously  analyzing  data, 
especially  preliminary  quarterly  reports,  and  are  experts  in  using  the  MoPerforms  analytical  tool. 

Missouri  Performs  (MoPerforms) 

The  DWD  Central  office  uses  a  custom  designed  software  tool  for  performance  analytics  called  MoPerforms. 
This  tool  provides  the  aggregation  of  all  variables  used  in  the  WIASRD  to  generate  the  required  reports  on  the 
Common  Measures;  while  DWD  staff  manually  submit  DOLETA  quarterly  reports.  It  is  possible,  however,  that 
MoPerforms  can  also  be  configured  for  the  cross-tabulation  and  analysis  of  any  of  the  WIASRD  variables  to 
create  management  reports  referred  to  as  decision  support  for  both  state  aggregate  and  each  local  region's 
data.  MoPerforms  preliminary  reports  may  also  be  used  to  verify  and  validate  observations  in  the  data. 

During  PY13,  MoPerforms  has  served  an  invaluable  role  in  permitting  Missouri  to  create  a  comprehensive  and 
frequently  available  report  series  that  permits  a  view  and  comparison  of  all  the  data  elements  required  by  the 
Civil  Rights  Center.  This  allows  Missouri  to  implement  analysis  and  subsequent  policy  in  Equal  Employment 
Opportunity  and  Affirmative  Action.  At  this  time,  Missouri  has  a  model  that  is  leading  the  nation  in  its  detail 
and  utility  for  this  purpose. 

Also,  added  to  MoPerforms  over  the  last  year:  a  comprehensive  array  of  reports  to  allow  an  analysis  of  Trade 
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Act  participants  and  program  outcomes;  enhancements  on  the  ability  to  develop  more  detailed  veterans 
reports  to  federal  and  state  veteran  staff  for  federal  auditing  and  local  veterans  staff  monitoring;  and  the 
aggregate  analysis  of  defining  the  long-term  unemployed  population  who  have  touched  the  workforce 
investment  system. 

Continuous  Improvement  and  Quality  Assurance 

DWD  Quality  Assurance  (QA)  implements  a  Continuous  Improvement  Review  program.  Missouri  has  been 
successful  in  getting  the  job  done  with  the  annual  Data  Element  Validation  required  by  DOLETA  using  QA  staff 
to  perform  both  desk  monitoring  of  data  elements  and  later  on-site  field  reviews.  To  supplement  not  only 
what  is  known  from  Toolbox  data,  the  QA  team  also  accesses  data  from  the  Department  of  Social  Services,  the 
Division  of  Employment  Security  and  the  Department  of  Revenue;  all  of  which  strengthens  their  ability  to 
streamline  validation  of  data. 

All  of  these  high  quality  staff  support  structures  and  program  actions  provide  the  assurance  of  uniform 
reporting  for  DQLETA.  From  the  WIASRD  data  in  Toolbox,  to  MoPerforms  and  the  analysis  of  individual  staff, 
the  JobStat  team  and  the  QA  unit  in  collaboration  with  the  Performance  Research  Unit,  Missouri  can  produce 
data  that  at  all  times  is  uniform  and  comparable  on  a  state-by-state  analysis. 


Performance  Achieved 

Next  Generation  Career  Centers 

It  is  essential  to  understanding  Missouri's  performance  reporting  to  first  provide  an  overview  of  the  Next 
Generation  Career  Center  (NGCC)  model.  The  implementation  and  configuration  of  serving  customers  under 
NGCC  has  impacted  Missouri  in  a  variety  of  ways,  most  dramatically  in  the  number  of  customers  served. 

In  July  2010,  the  Missouri  workforce  development  system  implemented  the  NGCC  integrated  skills-based 
service  delivery  model.  Approval  was  granted  by  DQLETA  for  the  use  of  a  streamlined  eligibility  determination 
and  documentation  process  for  a  two-year  pilot  program.  Instead  of  customers  separately  accounted  for  in 
one  WIA  program,  each  customer  was  enrolled  in  every  program  for  which  they  were  eligible  (Wagner-Peyser, 
WIA  Adult,  WIA  Dislocated  Worker,  etc).  This  process  meant  that  for  the  first  time  all  customer  registrations 
were  fully  accounted  for  as  having  been  provided  a  service.  Each  registration  constitutes  a  core  service  and 
subsequent  services,  being  more  intensive,  or  be  they  training,  determine  program  enrollment.  Youth  are  not 
co-enrolled  across  the  range  of  other  workforce  programs  primarily  because  of  their  difference  in  age  and  need; 
emphasizing  activities  more  related  to  skill  development,  or  education  than  work  experience. 

The  state  saw  a  major  increase  in  the  number  of  individuals  served  because  the  NGCC  model  removed  many 
artificial  barriers  to  the  immediate  access  to  WIA  funded  services.  In  March  2012  Missouri  was  given  approval 
to  have  the  pilot  extended  for  an  additional  two  years  because  the  NGCC  process  had  so  significantly 
increased  the  number  of  individuals  benefiting  from  WIA  funded  services  while  maintaining  acceptable  data 
element  validation  results.  Missouri  has  continued  to  show  positive  impacts  with  the  NGCC  pilot  program  and 
approval  was  granted  in  June  2014  to  extend  this  for  an  additional  program  year,  allowing  the  state  to 
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continue  using  the  NGCC  model  through  June  30,  2015. 

Table  1  displays  the  last  four  years  of  WIA  performance  in  Missouri  after  NGCC  for  comparison  to  the  program 
year  before  it  began  in  the  state.  The  most  ready  observation  of  the  data  reflects  the  significant  increase  in 
customers  served,  when  everyone  is  considered  a  participant,  minimally  eligible  for  a  core  service.  While  the 
change  is  structural  and  does  not  directly  reflect  increased  customer  traffic,  some  of  the  increase  was  traffic 
based  on  the  "Great  Recession's"  impact  to  Missouri,  beginning  in  2008.  Nevertheless,  the  count  for  Adult 
participants  being  served,  increased  by  285,040  from  9,715  in  PY09  to  294,755  in  PYIO,  an  increase  in  content 
by  29  times. 

Today,  the  most  significant  indicator  is  to  view  the  trend  since  implementation  of  NGCC.  Table  1  shows  a 
major  increase  for  two  years  and  then  a  decline  for  both  Adults  and  Dislocated  Workers.  In  the  Adult 
Program,  staff  assisted  have  declined  some  36,689  participants,  or  -10.6%  from  PYll  to  PY12  and  even 
greater,  by  68,097  or  -22%  over  the  last  program  year. 


Table  1:  Participants  Staff  Assisted  in  PY09-PY13  by  Program 


1 

Adult  Staff-Assisted 

Dislocated  Worker 
Staff-Assisted 

Youth  Staff-Assisted 

Participants  -  PY13 

240,674 

139,361 

4,637 

Participants  -  PY12 

308,771 

184,267 

5,442 

Participants -PYll 

345,460 

211,913 

5,697 

Participants -PYIO 

294,755 

186,441 

6,053 

Participants -PY09 

9,715 

13,990 

5,805 

Note:  Co-enrollment  prevents  the  sum  of  these  counts  from  being  an  accurate  total. 

Prepared  by  the  Performance  and  Research  Unit,  September  2014  -  Missouri  Division  of  Worl<force 
Development  -  Department  of  Economic  Development 


Participant  use  of  products  and  services  in  Missouri  is  very  high.  Table  2,  below,  indicates  that  the  percentage 
of  participants  served  who  used  a  product  or  service  has  remained  consistent  over  the  past  three  program 
years.  Of  the  total  243,434  served  in  PY13,  73.25%,  or  178,310  of  those  participants  used  a  product  or  service. 
Table  2,  page  8. 

Table  2  also  displays  product  and  service  use  by  various  credentials  that  were  attained.  The  lowest  rate  of  use 
came  from  the  group  of  participants  attaining  a  high  school  diploma/GED  credential  attainments.  Of  the  947 
participants  receiving  a  high  school  diploma/GED,  75.4%  used  a  product  or  service.  Of  the  457  participants 
who  received  an  Occupational  Skills  License,  437  (95.62%)  used  a  product  or  service.  The  clear  implication  is 
that  a  high  percentage  of  participants  in  Missouri  who  received  a  credential  took  advantage  of  the  products  or 
services  offered  in  the  state. 
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Table  2:  Participants  Served  Attaining  a  Credential  by  Products  and  Services  Used 


Participants  Served  PY11-PY13 

PYll 

PY12 

PY13 

Total  Served 

352,236 

312,829 

243,434 

Participants  Served  by  Products  and  Services  Use 

Total  That  Used  a  Product 

266,697 

226,666 

178,310 

75.72% 

72.46% 

73.25% 

Participants  Served  Attaining  a  Credential  by  Products  and  Services  Used 

Total  Receiving  High  School  Diploma/GED 

1,201 

1,580 

1,256 

Used  Products  and  Services 

679 

1,067 

947 

Percent  using  Products  and  Services 

56.54% 

67.53% 

75.40% 

Total  Receiving  AA  or  AS  Degree 

531 

571 

435 

Used  Products  and  Services 

292 

440 

397 

Percent  using  Products  and  Services 

54.99% 

77.06% 

91.26% 

Total  Receiving  BA  or  BS  Degree 

161 

224 

144 

Used  Products  and  Services 

89 

167 

123 

Percent  using  Products  and  Services 

55.28% 

74.55% 

85.42% 

Total  Receiving  Occupational  Skills  License 

583 

684 

457 

Used  Products  and  Services 

426 

627 

437 

Percent  using  Products  and  Services 

73.07% 

91.67% 

95.62% 

Total  Receiving  Occupational  Skills  Cert. 

1,803 

2,069 

1,521 

Used  Products  and  Services 

1,316 

1,910 

1,421 

Percent  using  Products  and  Services 

72.99% 

92.32% 

93.43% 

Total  Receiving  Other  Credential 

731 

495 

358 

Used  Products  and  Services 

483 

447 

344 

Percent  using  Products  and  Services 

66.07% 

90.30% 

96.09% 

Note:  Participants  Served  Adjusted  to  Exclude  Self-Served 

Prepared  by  the  Performance  and  Research  Unit,  September  2014  -  Missouri  Division  of  Workforce 
Development  -  Department  of  Economic  Development 
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Workforce  System  Performance  Data:  Common  Measures 

Missouri  has  met  or  exceeded  all  of  the  statewide  goals  required  of  it  in  PY13  by  attaining  at  least  80%  or 
more  of  its  negotiated  goal  in  each  measure  (see  Attachment  1  for  Missouri's  Statewide  WIA  Performance 
History).  From  PY09  through  PY13,  the  Adult  entered  employment  rate  achievements  were  no  less  than  83% 
and  as  high  as  86%  of  the  planned  performance  goals.  Results  are  equally  notable  for  the  Dislocated  Worker 
Program,  which  achieved  90%  or  greater  of  its  performance  goals  for  retention  from  PY09  through  PY13. 

Even  during  the  economic  downturn,  beginning  in  2008,  the  earnings  accomplishments  of  workforce  programs 
in  Missouri  has  been  admirable.  For  example.  Adult  average  earnings  achievements  have  been  as  high  as 
117%  of  their  goal  and  no  lower  than  94.8%  of  goal  the  last  five  program  years  (PY09-PY13).  For  the 
Dislocated  Worker  Program,  no  year  has  experienced  less  than  88%  of  goals  for  average  earnings,  and  for 
PYIO,  reached  as  high  as  131%  of  its  goal  attained. 

Twelve  of  the  fourteen  local  workforce  investment  boards  in  the  State  achieved  80%  or  better  of  their 
negotiated  performance  measures  in  PY13. 


Costs  of  Workforce  Investment  Activities 

Workforce  Investment  Costs  Relative  to  Performance 

As  noted  on  the  following  Tables  3,  4,  and  5,  the  costs  of  serving  participants  in  the  Adult,  Dislocated  Worker, 
and  Youth  programs  have  been  much  lower  since  NGCC  was  implemented  due  to  co-enrollment.  In  July  2013 
Missouri  passed  House  Bill  196,  which  eliminated  the  in-person  requirement  of  four  week  unemployment 
reporting  for  benefits.  This  law  change  has  made  an  impact  on  the  number  served  in  PY13. 


Table  3:  Adult  Program  Costs  per  Participant  Served 


Adult  Program  Costs  PY09  -  PY13 


PY09 

PYIO 

PYll 

PY12 

PY13 

Participants  J 
Served 

354,364 

402,772 

416,712 

398,424 

305,457 

Cost  Per^^H 
Served 

$61.95 

$35.65 

$28.61 

$32.67 

$32.45 

Total  Adult 
Expenses 

$21,953,601 

$14,357,172 

$11,923,658 

$13,016,551 

$9,910,925 
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Table  4:  Dislocated  Worker  Program  Costs  per  Participant  Served 

Dislocated  Worker  Program  Costs  PY09  -  PY13 


PY09 

PYIO 

PYll 

PY12 

PY13 

Participants 
Served 

13,957 

187,354 

212,749 

184,310 

139,361 

Cost  Per 
Served 

$1,963.56 

$77.29 

$58.44 

$65.71 

$65.80 

Total  DW  SI 
Expenses 

$27,405,472 

$14,481,501 

$12,433,844 

$12,111,443 

$9,169,324 

Table  5:  Youth  Program  Costs  per  Participant  Served 


Youth  Program  Costs  PY09  -  PY13 


PY09 

PYIO 

PYll 

PY12 

PY13 

Participants 
Served 

5,783 

6,163 

5,739 

5,442 

4,637 

Cost  Per 
Served 

$5,854.28 

$2,528.23 

$2,338.83 

$2,442.13 

$2,820.68 

Total  Youth 
Expenses 

$33,855,326 

$15,581,467 

$13,422,529 

$13,290,054 

$13,079,500 

State  Evaluations 

Gold  Standard  Evaluation 

In  February  2012,  DOLETA  implemented  a  five-year  WIA  Gold  Standard  Evaluation.  Missouri's  Central  Region 
Workforce  Investment  Board  was  randomly  selected  by  DOLETA  as  one  of  28  local  boards  across  the  nation  to 
participate  in  determining  the  "Gold  Standard"  for  WIA  Adult  and  Dislocated  Worker  programs.  Currently,  the 
contract  research  organization.  Social  Policy  Research  Associates,  is  in  the  final  stages  of  their  analysis  of  the 
data  gathered  during  this  study. 

As  of  September  18,  2013,  a  total  of  215  participants  in  the  Central  Workforce  Region  have  been  enrolled  into 
the  study.  The  study  focuses  on  how  well  WIA  services  are  working  in  the  local  region  and  how  they  can  be 
improved.  Customers  visiting  Central  Region  career  centers  (or  satellite  offices)  were  randomly  assigned  to 
one  of  three  research  groups:  Core;  Core  and  Intensive;  and  Full  WIA.  Once  a  client  is  randomly  assigned  to  a 
group,  they  remained  in  that  group  for  15  months  and  could  only  receive  services  allowed  for  that  group.  For 
example,  a  client  who  was  randomly  assigned  to  Core  could  only  receive  WIA  Core  services  for  15  months.  Of 
the  215  participants  in  the  study  so  far,  47  were  assigned  to  Core,  48  to  Core  and  Intensive,  and  215  to  full 


Missouri  WIA  Annual  Report  for  DOLETA  -  Program  Year  2013 


10 


WIA  services.  The  first  draft  of  the  consultant  report  is  anticipated  before  the  end  of  the  year. 

Return  on  Investment  (ROI)  from  the  Next  Generation  Career  Center  Model 

Since  Missouri  implemented  the  NGCC  model,  documentation  is  minimized  and  Career  Center  (core)  services 
are  available  to  virtually  any  job  seeker.  This  model  has  created  a  system  that  efficiently  uses  the  funding 
from  Wagner-Peyser  and  WIA  Adult,  and  Dislocated  Worker  programs  to  support  the  centers  and  their 
products  and  services.  Through  this  system,  a  simplified  calculation  of  the  Return-on-lnvestment  (ROI)  funds 
has  been  discovered. 

Assuming  that  "all"  of  the  funding  received  at  the  State  level  is  largely  used  to  support  the  NGCC  (except  for 
Youth  Program  funds)  and  can  be  considered  an  investment,  and  since  "o//"  of  the  Adult  Wagner-Peyser 
Program  customers  (staff-assisted)  can  be  considered  beneficiaries  of  that  investment,  it  becomes  a  matter  of 
simple  math  to  calculate  ROI.  Wagner-Peyser  Program  exiters  who  were  in  the  "Average  Earnings"  measure 
(those  exited  from  April  2012  to  March  of  2013  in  this  example)  amassed  total  earnings  of  $1,945,271,813.  To 
arrive  at  an  approximate  annualized  figure,  the  amount  can  be  simply  doubled.  (While  that  amount  is  not 
entirely  accurate,  for  purposes  of  maintaining  simplicity,  it  is  a  reasonable  presumption). 

The  next  step  in  the  calculation  is  to  determine  how  much  of  the  annualized  figure  can  be  considered  as  ROI. 
It  seems  logical  to  determine  that  these  individuals  would  now  be  taxpayers,  so  looking  at  an  average 
household  tax  rate  of  17%  (as  published  by  the  Internal  Revenue  Service)  a  return  of  $661,392,416  is 
produced.  Using  the  NGCC  model,  accounting  for  all  Wagner-Peyser,  WIA  Adult  and  Dislocated  Worker 
program  funds  received  by  the  state,  gives  an  annual  total  investment  of  $53,037,513.  This  calculation  yields  a 
return  (for  WIA  funds  and  Wagner-Peyser  funds)  of  $12.47  in  taxes  paid  back  to  the  system  for  every  dollar 
invested  in  the  Missouri  Career  Center  system  by  those  placed  in  sustained  employment. 

This  simplified  method  of  obtaining  ROI  would  indicate  that  the  NGCC  model  of  serving  everyone  with 
essentially  all  available  funds  appears  to  provide  an  efficient  and  effective  service  delivery  model  with  the 
potential  to  yield  admirable  results.  Moreover,  it  is  more  readily  apparent  and  understandable  to  the  public, 
especially  to  Congress  and  local  legislators. 


Customer  Satisfaction 

Customer  Satisfaction  for  Job  Seekers:  Statewide 

Missouri  is  one  of  several  states  in  compliance  with  USDOL  policy  in  reporting  the  Common  Measures.  As 
such,  it  does  not  report  an  overall  customer-satisfaction  measure  in  the  WIASRD. 

However,  a  newly  developed  job  seeker  customer  satisfaction  survey  was  put  in  place  and  implemented  after 
the  last  quarter  of  the  program  year.  This  survey  was  developed  from  a  modification  of  an  ordinal  scale  for 
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three  questions.  The  survey  was  distributed  via  e-mail  to  181,821  participants  using  a  system  called 
GovDelivery.com.  These  181,821  participants  can  be  described  as  anyone  provided  a  service  in  the  WIA  Adult 
program  between  July  1,  2013  to  June  30,  2014.  While  the  return  rate  for  the  total  surveys  distributed  was 
less  than  1%,  the  process  did  achieve  a  full  survey  response  from  620  participants  within  the  first  seven  days. 

The  following  questions  were  used  in  the  survey: 

•  Using  a  scale  of  1-5,  please  provide  a  rating  of  what  is  your  overall  satisfaction  with  the  services 
you  received  (?); 

•  Using  a  scale  of  1-5,  please  provide  a  rating  of  how  the  service  you  received  met  your 
expectations  (?);  and, 

•  Using  a  scale  of  1-5,  please  provide  a  rating  of  how  likely  you  are  to  return  to  the  Career  Center 
for  more  services  in  the  future? 

This  newly  developed  process  is  efficient,  easily  modified  and  useful  as  to  generating  real  results 
in  a  timely  manner.  However,  more  learning  about  the  implications  of  timing  the  sample  size  and 
distribution  will  be  needed  in  the  coming  months.  While  the  return  rate  on  the  survey  compared 
to  the  total  distributed  is  lower  than  hoped  for,  from  the  620  responses  we  learned  that: 

•  82.9%  of  all  of  those  Job  Seekers  responding  were  satisfied  with  the  services  received; 

•  84.8%  of  all  those  responding  felt  that  services  received  met  their  expectations  for  the 
service;  and 

•  Fully  90%  of  all  those  responding  would  be  likely  to  return  to  the  Career  Center  for  more 
services  in  the  future. 

Customer  Satisfaction  for  Job  Seekers:  St.  Louis  County 

An  admirable  effort  is  undertaken  on  an  ongoing  basis  by  staff  of  the  Board  managing  the  St.  Louis  County 
Workforce  Investment  region  to  survey  job  seekers.  In  Program  Year  2013,  an  average  of  17,500  job  seeker 
customers  in  its  two  American  Job  Centers  were  surveyed.  The  survey  was  delivered  via  "Survey  Monkey." 
Job  seekers  are  encouraged  to  complete  a  brief  survey  of  five  questions: 

1.  May  we  contact  you  by  phone  or  e-mail  at  a  later  date  to  follow-up 
if  necessary?  If  yes,  enter  the  information. 

2.  Please  select  area  seen: 

3.  Please  rate  the  service  you  received; 

4.  When  you  contacted  this  office,  what  type  of  help  were  you  seeking? 

5.  What  can  we  do  to  serve  you  better? 

For  PY13,  the  region  surveyed  job  seekers  from  June  1  until  October  20,  2014  and  received  an  average 
response  rate  of  13.9%  from  4,862  respondents.  The  results  for  this  process  are  very  positive.  Respondents 
were  affirmative  at  rates  greater  than  90%  for  service  features  they  had  experienced,  such  as  promptness  of 
service,  courtesy  and  knowledge  of  staff,  and  the  resources  provided  during  the  customer  visit.  Overall 
satisfaction  was  "yes"  for  more  than  90%  of  respondents. 
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St.  Louis  County  Board  staff  incorporates  the  customer  satisfaction  results  into  their  daily  operations  by  way  of 
analyzing  and  presenting  the  results  to  a  Customer  Satisfaction  Committee  of  the  Workforce  Investment 
Board. 

Program  Operation  Waivers 

During  PY13,  Missouri  had  only  one  waiver  to  implement,  which  was  the  Common  Measures  waiver.  Missouri 
has  implemented  this  waiver  statewide  and  continues  to  deliver  services  and  report  outcomes  through  the 
Common  Measures.  The  Common  Measures  include  service  categories  for  Adults,  Dislocated  Workers,  and 
Youth  customers.  For  each  group  there  are  placement,  retention,  and  earnings  measures,  resulting  in  the  nine 
reported  outcomes. 

Conclusions  for  the  Future 

Missouri  is  currently  researching  and  reviewing  the  Workforce  Innovation  and  Opportunity  Act  (WlOA).  The 
"opportunity"  provided  by  this  new  legislation  to  supplant  WIA  is  not  only  challenging,  but  welcomed 
because  it  will  be  so  timely.  Missouri's  Enterprise  database  and  customer  registration  system  must  be 
transformed  to  accommodate  new  customers  under  the  WlOA  and  integrate  into  its  performance  reporting. 
Since  many  of  the  underpinning  development  software  features,  such  as  programming  based  in  Oracle,  have 
run  their  useful  course  since  its  creation,  Missouri  will  be  pursuing  a  request  for  proposal  and  evaluating 
software  vendor  proposals  to  procure  a  Toolbox  replacement.  So,  with  WlOA  implementation  beginning  in 
July  2015,  this  will  be  an  opportune  time  for  a  new  system  that  will  integrate  reporting  for  new  partner 
agencies.  The  Toolbox  replacement  challenge  will  be  in  maintaining  the  WIASRD  nature  of  the  database, 
while  also  seeking  the  analytical  capacity  of  MoPerforms. 

Missouri  remains  committed  to  the  integrated  nature  of  WlOA  service  delivery  and  believes  it  has  the  ability 
to  create  an  exemplary  model  of  implementation  based  on  our  prior  experiences  with  the  Next  Generation 
Career  Center  model.  The  concept  of  WlOA  "registering  and  co-enrolling  participants"  is  just  one  example  of 
integrated  tracking  that  is  believed  to  enable  an  effective  reporting  of  both  performance  outcomes  and 
continued  use  of  decision-based  management  tools. 

Attachment  1  (follows) 

Statewide  Performance  History 
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